<HTML>

<BODY BGCOLOR="white">

<center>
<FONT SIZE=+2 COLOR="#BB0000">CS346 - Spring 2014<BR>Database System Implementation</FONT>
</center>

<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>RedBase Part 3: The System Management Component</b></center>
<center><b>Due Sunday May 11</b></center>
</font></td></tr>
</table>

<ul>
<li> <a href="#intro">Introduction</a>
<li> <a href="#utilities">Command Line Utilities</a>
<li> <a href="#system">RedBase System Commands</a>
<li> <a href="#interface"><b>SM Interface</b></a>
<li> <a href="#error">Return Codes and Error Handling</a>
<li> <a href="#metadata">Metadata Management</a>
<li> <a href="#setup">Setup and Files</a>
<li> <a href="#parser">The Parser</a>
<li> <a href="#printer">The Printer Class</a>
<li> <a href="#additional">Additional Commands</a>
<li> <a href="#misc">Documentation, Testing, Submission, Etc.</a>
</ul>

<a name="intro">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>Introduction</b></center>
</font></td></tr>
</table>

The third part of the RedBase system you will implement is the
<i>System Management</i> (<i>SM</i>) component.  This component
provides a "potpourri" of capabilities:

<ol>

<li> <b>Unix command line utilities</b> - for creating and destroying
RedBase databases, invoking the system

<p><li> <b>Data definition language (DDL) commands</b> - for creating
and dropping relations, creating and dropping indexes

<p><li> <b>System utilities</b> - for bulk loading, help, printing
relations, setting parameters

<p><li> <b>Metadata management</b> - for maintaining system catalogs

</ol>

Since command line utilities are invoked at the Unix level, you will
write stand-alone programs for each of them.  DDL commands and system
utilities are invoked from a RedBase prompt.  You don't need to worry
about managing the prompt, user interaction, or command parsing -- we
provide a parser that handles user interaction and recognizes all
RedBase commands.  The parser invokes methods implemented by your SM
and later QL (Part 4) components.  <a href="#parser">More details on
the parser</a> are given below.  Metadata management is internal to
the system.

<p>Throughout this document we assume (and strongly suggest) a design
in which each RedBase relation is stored in its own RM component file,
and each tuple of a RedBase relation is stored as a record in the
appropriate file.  Consequently, "files" are now sometimes referred to
as "relations," and "records" are now referred to as "tuples."

<a name="utilities">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>Command Line Utilities</b></center>
</font></td></tr>
</table>

You should implement the following three utilities:

<ul>
<li><tt> dbcreate</tt> <i>DBname</i>
<li><tt> dbdestroy</tt> <i>DBname</i>
<li><tt> redbase</tt> <i>DBname</i>
</ul>

Separate stand-alone programs should be written for each of these
utilities.  You should create executable files and name them as
<tt>dbcreate</tt>, <tt>dbdestroy</tt>, and <tt>redbase</tt>, so that
the utilities can be invoked from the Unix command prompt.  We are
providing program "shells" for two of these commands, which you can
use as starting points for your implementation.  They are located in
<tt>redbase.cc</tt> and <tt>dbcreate.cc</tt>, obtained when you run
the <tt>setup</tt> script with argument "<tt>3</tt>" (for project part
3) -- there's more on <a href="#setup">setup for this project part</a>
described below.

<p><i>DBname</i> is a user-specified name associated with a particular
database.  Your RedBase system should be able to manage multiple
databases, although it is invoked (using the <tt>redbase</tt> command)
for only one database at a time.  All of the files associated with a
given database <i>DBname</i> -- files containing relations, indexes,
and metadata -- should be stored in a subdirectory for that database;
the subdirectory can be called <i>DBname</i>.  Command
<tt>dbcreate</tt> should set up the subdirectory.  Command
<tt>dbdestroy</tt> should destroy the subdirectory and all of its
contents.  One of the first things command <tt>redbase</tt> should do
is change the current working directory to the subdirectory for the
specified <i>DBname</i>.

<h4> The "dbcreate" Command </h4>

The <tt>dbcreate</tt> command is invoked with the name of the new
database.  The following <i>approximate</i> code segment implements
this command:

<pre>
   main (
    int argc,      // length of array argv
    char **argv )  // argv is an array of pointers
                   // argv[0] points to the name of the command
                   // argv[1] points to argument DBname
   {
      char *dbname;
      char command[80] = "mkdir ";

      if (argc != 2) {
         cerr << "Usage: " << argv[0] << " dbname \n";
         exit(1);
      }

      // The database name is the second argument
      dbname = argv[1];

      // Create a subdirectory for the database
      system (strcat(command,dbname));

      if (chdir(dbname) < 0) {
         cerr << argv[0] << " chdir error to " << dbname << "\n";
         exit(1);
      }

      // Create the system catalogs
      ...
   }
</pre>

<h4> The "dbdestroy" Command</h4>

The <tt>dbdestroy</tt> command is invoked with the name of the database
to be destroyed.  The code for <tt>dbdestroy</tt> is similar in spirit to
the code for <tt>dbcreate</tt>.  In <tt>dbdestroy</tt>, the following system
call should be used:

<pre>
   char command[80] = "rm -r ";
   system(strcat(command, dbname));
</pre>

You don't need to worry about individually removing each file for
<i>DBname</i> -- all files in the subdirectory will be removed when
you invoke <tt>rm</tt> with the <tt>-r</tt> option.

<h4> The "redbase" Command </h4>

The <tt>redbase</tt> command is invoked with the name of the database to
be used for the RedBase session.  This command will now be the main
driver of your RedBase system.  The following approximate (and
incomplete) code segment is used in the implementation of this
command:

<pre>
   main(...)
    {
      ...
      // initialize RedBase components
      PF_Manager pfm;
      RM_Manager rmm(pfm);
      IX_Manager ixm(pfm);
      SM_Manager smm(ixm, rmm);
      QL_Manager qlm(smm, ixm, rmm);
      // open the database
      if (rc = smm.OpenDb(dbname)) ...
      // call the parser
      RBparse(pfm, smm, qlm);
      // close the database
      if (rc = smm.CloseDb()) ...
    }
</pre>

Methods <tt>SM_Manager::OpenDb</tt> and <tt>SM_Manager::CloseDb</tt>
are described in the <a href="#interface">SM interface</a> below.  We
are providing stub files defining the <tt>QL_Manager</tt> class (more
on this later).  You do not need to write routine <tt>RBparse</tt> --
it is provided by the parser (see <tt>parse.y</tt> if you're
interested).  When <tt>RBparse</tt> is called, a loop repeatedly
prompts the user for a command and then calls appropriate SM or QL
methods to process the command.  The command loop terminates and the
call to <tt>RBparse</tt> returns when the user types "<tt>exit;</tt>"
at the parser prompt.  (Termination also occurs if a negative error
code is returned from an SM or QL method call, or if an end-of-file
character is generated accidentally.)

<a name="system">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>RedBase System Commands</b></center>
</font></td></tr>
</table>

Once the RedBase system has been started up, the user submits commands
that are classified into data definition language (DDL) commands,
system utility commands, and data manipulation language (DML)
commands.  You will implement DDL commands and system utilities in
this component.  DML commands will be implemented in Part 4 of the
project.  All RedBase command names are case-insensitive, and all
commands are terminated by a semicolon.  (It's a common mistake among
RedBase users to forget to type the semicolon.)

<p>To describe the DDL and system utility commands, we'll first
specify the syntax of the commands and explain what they should do
from a user's perspective.  Then we'll specify the SM component
interface, which includes the methods that support the commands, as
well as methods <tt>OpenDb</tt> and <tt>CloseDb</tt>.

<h4> DDL Commands </h4>

<ol>

<li> <tt>create table</tt> <i>relName</i><tt>(</tt><i>attrName1 Type1</i>, 
      <i>attrName2 Type2</i>, ...,
      <i>attrNameN TypeN</i><tt>);</tt>

<p>The <tt>create table</tt> command creates a relation with the
specified name and schema.  Relation and attribute names are limited
to <tt>MAXNAME = 24</tt> characters each, and must begin with a
letter.  Within a given database every relation name must be unique,
and within a given relation every attribute name must be unique.
(However, relation names may be duplicated across databases, and
attribute names may be duplicated across relations.)  Every relation
must have at least one and no more than <tt>MAXATTRS = 40</tt>
attributes.  The syntax of each <i>Type</i> argument is a
one-character type specification -- "<tt>i</tt>", "<tt>f</tt>", or
"<tt>c</tt>", for integer, float, or character string, respectively --
followed by an integer length.  For "<tt>i</tt>" and "<tt>f</tt>" the
only valid length is 4; for "<tt>c</tt>" valid lengths range from 1 to
<tt>MAXSTRINGLEN = 255</tt>.  As an example, "<tt>c50</tt>" is the
type specification for an attribute whose values are character strings
of length 50.  Note that when the type specification
"<tt>c</tt><i>N</i>" is given, the attribute must be able to store
strings of length <i>N</i> -- the system should not use one of the
characters for null termination.  The constants <tt>MAXNAME</tt>,
<tt>MAXATTRS</tt>, and <tt>MAXSTRINGLEN</tt> are defined in
<tt>redbase.h</tt>.

<p><li> <tt>drop table</tt> <i>relName</i><tt>;</tt>

<p>The <tt>drop table</tt> command destroys relation <i>relName</i>,
along with all indexes on the relation.

<p><li> <tt>create index</tt> <i>relName</i><tt>(</tt><i>attrName</i><tt>);</tt>

<p>The <tt>create index</tt> command creates an index on attribute
<i>attrName</i> of relation <i>relName</i> and builds the index for
the current tuples in the relation.  Only one index may be created for
each attribute of a relation.

<p><li> <tt>drop index</tt> <i>relName</i><tt>(</tt><i>attrName</i><tt>);</tt>

<p>The <tt>drop index</tt> command destroys the index on attribute
<i>attrName</i> of relation <i>relName</i>.

</ol>

<h4> System Utilities </h4>

System utility commands are invoked from the RedBase prompt, just like
DDL (and later DML) commands.

<ol>

<li> <tt>load</tt> <i>relName</i><tt>("</tt><i>FileName</i><tt>");</tt>

<p>The <tt>load</tt> utility performs bulk loading of the named
relation from the specified Unix file: all tuples specified in the
load file are inserted into the relation.  The <i>FileName</i> should
be a complete Unix path enclosed in quotes (e.g.,
"<tt>/usr/class/cs346/redbase/data/student.data</tt>").  This file holds
the data to be loaded into the relation in ASCII format.  Every tuple
to be inserted is on a separate line in the file, with attribute
values separated by commas and appearing in the same order as in the
<tt>create table</tt> command that was executed for relation
<i>relName</i>.

<p>Integer attribute values are specified in the ASCII load file as,
e.g., <tt>10</tt> or <tt>-5</tt>, float values are specified as, e.g.,
<tt>3.5E-3</tt>, and you may assume these values are in the right
format. Character string values are specified as, e.g., <tt>Smith</tt>
(without quotes).  You may assume that character string attribute
values will not contain commas.  Character strings in the load file
can be of any length up to the length specified for the corresponding
attribute, including zero length (no characters for that field in the
load file).  If a character string is too long, you may silently
truncate it, or you may generate a nonzero return code and stop
loading, whichever behavior you prefer.

<p>Some example load files can be found in directory
<tt>/usr/class/cs346/redbase/data/</tt>.  We strongly suggest that you
create your own additional data files for loading, as well as for
future testing and experimentation.  If you would like to share your
data files with the class, you may place them in directory
<tt>/usr/class/cs346/redbase/data/</tt>, as discussed in the <a
href="logistics.html">RedBase
Logistics</a> document.

<p><li> <tt>help</tt> [<i>relName</i>]<tt>;</tt>

<p>The square brackets here are not part of the command syntax -- they
indicate that <i>relName</i> is optional.  If a <i>relName</i> is not
specified, then the help utility prints the names of all relations in
the database.  (You may include additional information if you like.)
If a <i>relName</i> is specified, then the help utility prints the
name, type, length, and offset of each attribute in the specified
relation, together with any other information you feel may be useful.

<p><li> <tt>print</tt> <i>relName</i><tt>;</tt>

<p>The <tt>print</tt> utility displays the current set of tuples in
relation <i>relName</i>.

<p><li> <tt>set</tt> <i>Param</i> <tt>= "</tt><i>Value</i><tt>";</tt>

<p>The <tt>set</tt> utility allows the user to set system parameters
without needing to recompile the system, or even exit the RedBase
prompt.  You should determine if there are any parameters you might
find useful to control in this manner.  (Examples might be level of
tracing information produced, debugging flags, etc.)  You are not
required to implement <tt>set</tt> for any particular parameters;
we're just providing the necessary "hooks" in case you find it
convenient to do so.

</ol>

<a name="interface">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>SM Interface</b></center>
</font></td></tr>
</table>

When the parser recognizes a DDL or system utility command, it calls a
public method of the <tt>SM_Manager</tt> class, described below.
<tt>SM_Manager</tt> is the only class in the SM interface.  The SM
interface also includes an <tt>SM_PrintError</tt> routine for printing
messages associated with nonzero SM return codes.  An initial header
file for the SM interface is in <tt>sm.h</tt>, obtained via the
<tt>setup</tt> script.  An additional SM "stub" file contains a shell
of all of the public methods of the SM class.  You should rename
<tt>sm_stub.cc</tt> as <tt>sm_manager.cc</tt> and complete the
implementation.  As usual, all SM component public methods (except
constructors and destructors) should return 0 if they complete
normally and a nonzero return code otherwise.  <a
href="#error">Parser handling of nonzero return codes</a> is
discussed below.

<h4> SM_Manager Class </h4>

<ul>

<li> As always, all necessary component initialization should take
place within the constructor for the <tt>SM_Manager</tt> class, and
all "cleanup" should take place within its destructor.

<p><li> When implementing the public methods of the <tt>SM_Manager</tt>
class that correspond to commands, in addition to adhering to the
method descriptions below, please refer back to the <a
href="#system">command descriptions</a> given earlier.

<p><li> As usual, you are free to extend the interface if you find it
convenient, and in fact we make a recommendation below on one way to
extend it.  You should not modify the existing methods however, since
they are called by the parser as specified.

<p><li> For user-specified names -- names of relations,
attributes, files, etc. -- it is up to you whether you want to treat
the names as case-sensitive or case-insensitive.  All user-specified
names are passed from the parser to the SM component methods exactly
as they are typed by the user.

<p><li> Before specifying the interface for the <tt>SM_Manager</tt>
class, we define two structures: <tt>AttrInfo</tt> and
<tt>DataAttrInfo</tt>.  <tt>AttrInfo</tt> is used for a parameter to
<tt>SM_Manager::CreateTable</tt>, and <tt>DataAttrInfo</tt> is used
for a parameter to the <tt>Printer</tt> class (<a
href="#printer">described below</a>).  <tt>AttrInfo</tt> is defined in
<tt>parser.h</tt> and <tt>DataAttrInfo</tt> is defined in
<tt>printer.h</tt>.  You will almost certainly find these structures
useful elsewhere within your implementation.

</ul>

<pre>

   // Used by SM_Manager::CreateTable
   struct AttrInfo {
      char     *attrName;           // Attribute name
      AttrType attrType;            // Type of attribute
      int      attrLength;          // Length of attribute
   };

   // Used by Printer class
   struct DataAttrInfo {
      char     relName[MAXNAME+1];  // Relation name
      char     attrName[MAXNAME+1]; // Attribute name
      int      offset;              // Offset of attribute 
      AttrType attrType;            // Type of attribute 
      int      attrLength;          // Length of attribute
      int      indexNo;             // Attribute index number
   };


   class SM_Manager {
     public:
          SM_Manager  (IX_Manager &ixm, RM_Manager &rmm);  // Constructor
          ~SM_Manager ();                                  // Destructor
       RC OpenDb      (const char *dbName);                // Open database
       RC CloseDb     ();                                  // Close database
       RC CreateTable (const char *relName,                // Create relation
                       int        attrCount,
                       AttrInfo   *attributes);
       RC DropTable   (const char *relName);               // Destroy relation
       RC CreateIndex (const char *relName,                // Create index
                       const char *attrName);
       RC DropIndex   (const char *relName,                // Destroy index
                       const char *attrName);
       RC Load        (const char *relName,                // Load utility
                       const char *fileName);
       RC Help        ();                                  // Help for database
       RC Help        (const char *relName);               // Help for relation
       RC Print       (const char *relName);               // Print relation
       RC Set         (const char *paramName,              // Set system parameter
                       const char *value);
   };
</pre>

<h4>
RC OpenDb (const char *dbName)
</h4>

Recall that this method, along with method <tt>CloseDb</tt>, is called
by your code implementing the <tt>redbase</tt> command line utility.
This method should change to the directory for the database named
<tt>*dbName</tt> (using system call <tt>chdir</tt>), then open the
files containing the system catalogs for the database.  <a
href="#metadata">System catalogs are described below</a>.

<h4>
RC CloseDb ()
</h4>

This method should close all open files in the current database.
Closing the files will automatically cause all relevant buffers to be
flushed to disk.

<h4>
RC CreateTable (const char *relName, int attrCount, AttrInfo *attributes)
</h4>

This method creates a new relation named <tt>*relName</tt>.  Argument
<tt>attrCount</tt> indicates the number of attributes in the relation
(between 1 and <tt>MAXATTRS</tt>).  Argument <tt>attributes</tt> is an
array of length <tt>attrCount</tt>.  For the <i>i</i>th attribute of
the new relation, the <i>i</i>th element of array <tt>attributes</tt>
contains the name, type, and length of the attribute (see definition
of <tt>AttrInfo</tt> above).

<p>This method should first update the system catalogs: a tuple for
the new relation should be added to a catalog relation called
<tt>relcat</tt>, and a tuple for each attribute should be added to a
catalog relation called <tt>attrcat</tt>.  (Catalogs <tt>relcat</tt>
and <tt>attrcat</tt> describe all relations and attributes in the
database, respectively; <a href="#metadata"> details are given
below</a>.)  To create entries for the new relation in the catalogs,
you will need to calculate tuple length and attribute offset
information from the arguments passed to this method.  After updating
the catalogs, method <tt>RM_Manager::CreateFile</tt> should be called
to create a file that will hold the tuples of the new relation.

<h4>
RC DropTable (const char *relName)
</h4>

This method should destroy the relation named <tt>*relName</tt> and
all indexes on that relation.  The indexes are found by accessing
catalog <tt>attrcat</tt>, and the index files are destroyed by calling
method <tt>IX_Manager::DestroyIndex</tt>.  The file for the relation
itself is destroyed by calling method
<tt>RM_Manager::DestroyFile</tt>.  Information about the destroyed
relation should be deleted from catalogs <tt>relcat</tt> and
<tt>attrcat</tt>.

<h4>
RC CreateIndex (const char *relName, const char *attrName)
</h4>

This method should create an index on attribute <tt>*attrName</tt> of
relation <tt>*relName</tt> and build the index from the current
contents of the relation.  This method should first check (by
accessing catalog <tt>attrcat</tt>) that there is not already an index
on the specified attribute; if there is, a nonzero code should be
returned.  Catalog <tt>attrcat</tt> should be updated to reflect the
new index, then method <tt>IX_Manager::CreateIndex</tt> should be
called to create the index.  Building the index consists of: (1)
opening the index; (2) using RM component methods to scan through the
records to be indexed, repeatedly calling
<tt>IX_IndexHandle::InsertEntry</tt>; (3) closing the index.

<h4>
RC DropIndex (const char *relName, const char *attrName)
</h4>

This method should destroy the index on attribute <tt>*attrName</tt>
of relation <tt>*relName</tt>.  This method should first check (by
accessing catalog <tt>attrcat</tt>) that the index exists; if not, a
nonzero code should be returned.  Catalog <tt>attrcat</tt> should be
updated, and method <tt>IX_Manager::DestroyIndex</tt> should be called
to destroy the index.

<h4>
RC Load (const char *relName, const char *fileName)
</h4>

This method should insert into the relation named <tt>*relName</tt>
all tuples in the Unix ASCII file named <tt>*fileName</tt>.  The
tuples are formatted as described earlier.  Method
<tt>RM_Manager::OpenFile</tt> is called to open the relation file and
method <tt> IX_Manager::OpenIndex</tt> is called to open each index.
Then the tuples are read from the ASCII file one at a time, using
schema information obtained from catalog <tt>attrcat</tt>.  After
reading each tuple, method <tt>RM_FileHandle::InsertRec</tt> is called
to insert the tuple into the relation, and method
<tt>IX_IndexHandle::InsertEntry</tt> is called for each index to make
appropriate index entries for the tuple.  If an error occurs during
loading, this method should just return a nonzero code -- do not worry
about leaving relations or indexes in a "half-loaded" state.  After
all the tuples are loaded, all opened files, including the ASCII load
file, should be closed.  Note that the ASCII load file is manipulated
directly, not using the RedBase file interface.

<h4>
RC Help ()
</h4>

If <tt>Help</tt> is called with no arguments, then a list of all
relations should be obtained and printed by scanning catalog
<tt>relcat</tt>.  For uniformity, this method must do all of its
printing using the <tt>Printer</tt> class that we are providing, <a
href="#printer">described below</a>.

<h4>
RC Help (const char *relName)
</h4>

If <tt>Help</tt> is called with argument <tt>*relName</tt>, then
information about the attributes in the named relation should be
obtained and printed by accessing catalog <tt>attrcat</tt>.  For
uniformity, this method must do all of its printing using the
<tt>Printer</tt> class that we are providing, <a
href="#printer">described below</a>.

<h4>
RC Print (const char *relName)
</h4>

This method should open the relation named <tt>*relName</tt>, print
its contents by scanning the entire relation, print the total number
of tuples returned, then close the relation.  For uniformity, this
method must do all of its printing using the <tt>Printer</tt>
class that we are providing, <a href="#printer">described below</a>.

<h4>
RC Set (const char *paramName, const char *value)
</h4>

This method is called when the user requests to set a system
parameter, as described earlier.  This method should set the parameter
identified by argument <tt>paramName</tt> to the value specified by
argument <tt>*value</tt>.  Regardless of the type of the system
parameter (integer, character, etc.), the <tt>value</tt> is passed to
this method as a string, exactly as typed by the user (without the
quotes).

<h4> SM_PrintError</h4>

<h4>
void SM_PrintError (RC rc);
</h4>

This routine should write a message associated with the nonzero SM
return code <tt>rc</tt> onto the Unix <tt>stderr</tt> output stream.
This routine has no return value.

<a name="error">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>Return Codes and Error Handling</b></center>
</font></td></tr>
</table>

Return codes and error handling will generally continue in the style you
adopted for Parts 1 and 2 of the project.  If the parser receives a
nonzero return code after calling one of your SM component methods, it
will use the value of the return code to call the appropriate
component's <tt>PrintError</tt> routine.  If the code is positive, the
parser will then resume with the command loop under the assumption
that the database is intact and processing can continue.  If the code
is negative, the parser will terminate the command loop.

<p><b>Note:</b> You should not call <tt>SM_PrintError</tt>
directly from within the SM (or any other) component.
<tt>SM_PrintError</tt> is called automatically by the parser when it
receives a nonzero return code from an SM component method it invokes.
You may, however, need to call <tt>SM_PrintError</tt> from your
stand-alone programs implementing the <a href="#utilities">command line
utilities</a> <tt>dbcreate</tt>, <tt>dbdestroy</tt>, and
<tt>redbase</tt>.

<a name="metadata">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>Metadata Management</b></center>
</font></td></tr>
</table>

The internal schema information that describes the database relations,
the format of their attributes, and which attributes have indexes, is
stored in special system-managed relations called <i>catalogs</i>.
Like all relations, files for the catalog relations can and should be
manipulated using the methods you implemented in Parts 1 and 2.

<p>For the basic project you will need only two catalogs:
<tt>relcat</tt> and <tt>attrcat</tt>.  <i>Please do not change the
names of these catalogs.</i> These two catalog relations should be
created when the "<tt>dbcreate</tt> <i>DBname</i>" command is
executed, and they should be stored in the subdirectory called
<i>DBname</i> (each database has its own catalogs).  Since you will
create and manipulate these relations using RM and perhaps IX
component methods, the stand-alone code you write for the
<tt>dbcreate</tt> command must be linked with your RM and IX
components.

<p>The <tt>relcat</tt> relation is used to keep track of all relations
in the database.  There is one tuple in <tt>relcat</tt> for each
relation.  Examples of the kind of information you might choose to
include in each tuple of the <tt>relcat</tt> relation are:

<p>
<table border=1>
<tr><td> <i>relName</i>       <td> relation name
<tr><td> <i>tupleLength</i>   <td> tuple length in bytes
<tr><td> <i>attrCount</i>     <td> number of attributes
<tr><td> <i>indexCount</i>    <td> number of indexed attributes
</table>

<p>The actual information in <tt>relcat</tt> may vary from design to
design -- you should include all information that you find useful.

<p>The <tt>attrcat</tt> relation is used to keep track of all
attributes of all relations in the database.  There is one tuple in
<tt>attrcat</tt> for each attribute of each relation.  Examples of the
kind of information you might choose to include in each tuple of the
<tt>attrcat</tt> relation are:

<p>
<table border=1>
<tr><td> <i>relName</i>   <td> this attribute's relation
<tr><td> <i>attrName</i>  <td> attribute name
<tr><td> <i>offset</i>    <td> offset in bytes from beginning of tuple
<tr><td> <i>attrType</i>  <td> attribute type
<tr><td> <i>attrLenth</i> <td> attribute length
<tr><td> <i>indexNo</i>   <td> index number, or <tt>-1</tt> if not indexed
</table>

<p>Again, the actual information in <tt>attrcat</tt> may vary from
design to design, and you should include all information that you find
useful.

<p>You should enable the user to access the catalog relations using
standard commands, as if the catalogs were regular database relations.
To do so, you will need to insert descriptions for <tt>relcat</tt> and
<tt>attrcat</tt> into <tt>relcat</tt> and <tt>attrcat</tt> when you
create them.  Users should not, however, be permitted to load tuples
into or drop either of the system catalogs.

<p>Because the system catalogs are accessed very frequently, we
suggest that you open the catalogs for a database when the database is
opened, then keep the catalogs open until the database is closed.  One
effect of doing so is that updates to the catalogs may not be
reflected onto disk immediately.  Thus, if you open a catalog a second
time (to implement the <tt>help</tt> utility, for example, or to
<tt>print</tt> the contents of a catalog), then you may not see the
most current version of the catalog.  One solution to this problem is
to call <tt>RM_FileHandle::ForcePages</tt> each time a catalog is
changed -- don't forget to also flush any header information, and to
call <tt>IX_FileHandle::ForcePages</tt> for any indexes you've created
on the catalog.

<h4> Metadata Methods </h4>

You will almost certainly find it convenient for this component and
for Part 4 to implement methods that obtain metadata information from
catalogs <tt>relcat</tt> and/or <tt>attrcat</tt> for a specific
relation or attribute.  We have not included these methods in our
interface for <tt>SM_Manager</tt> since the type of information
returned may be dependent on how you organize your metadata.  However,
we do highly recommend that you define and implement such methods now,
both for clean coding in the SM component and to save yourself extra
work when you attack the QL component.

<a name="setup">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>Setup and Files</b></center>
</font></td></tr>
</table>

Before beginning your work on the SM component you should run the
<tt>setup</tt> script with argument "<tt>3</tt>" (for project part 3).
The script will copy or link more than 20 files.  Most of these files
are for the parser, but we are also including "stubs" for the SM and
QL components.  The first thing you should do after running
<tt>setup</tt> is to uncomment SM_SOURCES, QL_SOURCES, UTIL_SOURCES,
and PARSER_SOURCES in <tt>Makefile</tt> to build
the parser (<tt>libparser.a</tt>) and the stubs for the SM and QL
components.  To implement the SM component you will be replacing the
SM stub and its temporary library (<tt>libsm.a</tt>) with your own.
You will continue to use the QL stub (and <tt>libql.a</tt>) until
project part 4.  The QL stub is necessary so that executables
involving the parser link properly.  The files provided also include
<tt>printer.h</tt> and <tt>printer.cc</tt>, which contain the
<tt>Printer</tt> class <a href="#printer">described below</a>.

<a name="parser">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>The Parser</b></center>
</font></td></tr>
</table>

The RedBase parser takes a command from the user, parses it, then
calls an SM or QL component method to execute the command.  After
command execution is done, the parser prompts the user for a new
command.  For convenience, the parser also accepts Unix shell commands
preceded by an exclamation mark ("<tt>!ls</tt>", for example), so you
don't need to exit the RedBase system in order to do things such as
see what your directory looks like.  To exit the parser's command
loop, type "<tt>exit;</tt>" at the prompt.

<tt>parser_test</tt> allows you to experiment with RedBase prompt
interaction and see in general how the parser operates.  Remember that
your <tt>redbase.cc</tt> needs to call function <tt>RBparse</tt>, so
you must include <tt>parser.h</tt> in <tt>redbase.cc</tt>.  Finally,
library <tt>libql.a</tt> is necessary for any linking step that
includes the parser.

<p>The parser is implemented using <i>flex</i> and <i>yacc</i>.  If
you would like to modify the parser, the file called
<tt>Parser.HowTo</tt> explains how to get started.

<a name="printer">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>The Printer Class</b></center>
</font></td></tr>
</table>

We are providing a class for printing tuples of a relation one at a
time.  This class <i>must</i> be used by your
<tt>SM_Manager::Help</tt> and <tt>SM_Manager::Print</tt> methods, and
it will be required for some QL component methods as well.

<pre>
   class Printer {
     public: 
       Printer(const DataAttrInfo *attributes, const int attrCount);
       ~Printer();         
       void PrintHeader(ostream &c) const;
       Void Print(ostream &c, const char * const data);
       void Print(ostream &c, const void * const data[]);
       void PrintFooter(ostream &c) const;
   };
</pre>

<h4>
Printer(const DataAttrInfo *attributes, const int attrCount);
</h4>

The constructor takes a pointer to <tt>attrCount</tt> elements of
<tt>DataAttrInfo</tt>, describing the schema of the tuples to be
printed.  The <tt>DataAttrInfo</tt> is used to display the header
information and the tuples in a nice fashion.

<h4>
void PrintHeader(ostream &c) const;
</h4>

This method prints header information.  Each attribute name is
printed, along with its relation name if needed to disambiguate the
attribute.  A row of dashes is then printed.  This method should be
called once, before printing any actual tuples.

<h4>
void Print(ostream &c, const char * const data);<br>
void Print(ostream &c, const void * const data[]);
</h4>

This method prints the tuple that is pointed to by <tt>char*
data</tt>.  For the QL component, this method also can receive an
array of <tt>void*</tt> pointers, where each one points to an
attribute value.  (The utility of the second version will become
evident when implementing the QL component.)

<h4>
void PrintFooter(ostream &c) const;
</h4>

This method prints "footer" information about the number of tuples
printed.  This method should be called once, after the last tuple has
been printed.

<p>As an example of how the <tt>Printer</tt> class is used, the
following code segment is similar to what you will write in your
<tt>SM_Manager::Print</tt> method.  Note that this code is simplified,
and you will need to fill in the details.

<pre>
   DataAttrInfo *attributes;
   int attrCount;
   RM_FileHandle rfh;
   RM_Record rec;
   char *data;

   // Fill in the attributes structure, define the RM_FileHandle
   ...

   // Instantiate a Printer object and print the header information
   Printer p(attributes, attrCount);
   p.PrintHeader(cout);

   // Open the file and set up the file scan
   if ((rc=rmm->OpenFile(relName, rfh)))
      return(rc);

   RM_FileScan rfs;

   if ((rc = rfs.OpenScan(rfh, INT, sizeof(int), 0, NO_OP, NULL))) 
      return (rc);

   // Print each tuple
   while (rc!=RM_EOF) {
      rc = rfs.GetNextRec(rec);

      if (rc!=0 && rc!=RM_EOF)
         return (rc);

      if (rc!=RM_EOF) {
         rec.GetData(data);
       p.Print(cout, data);
      }
   }

   // Print the footer information
   p.PrintFooter(cout);

   // Close the scan, file, delete the attributes pointer, etc.
   ...
</pre>

<a name="additional">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>Additional Commands</b></center>
</font></td></tr>
</table>

In addition to the DDL commands and system utilities described
earlier, and the DML commands you will implement in the QL component,
we have exposed some administrative commands in the parser.  You will
find these commands particularly useful for testing and tuning your
system's I/O performance, i.e., to gear up for the efficiency contest
which is now only two components away.  The commands are:

<p>
<table border=1>
<tr><td> <tt>reset buffer;</tt>         <td> Remove all unpinned pages from the buffer pool
<tr><td> <tt>print buffer;</tt>         <td> Display simple information about pages in the buffer
<tr><td> <tt>resize buffer</tt> <i>i</i><tt>;</tt>  <td> Resize the buffer pool to <i>i</i> pages (<i>i</i> an integer)
<tr><td> <tt>print io;</tt>             <td> Display the I/O statistics
<tr><td> <tt>reset io;</tt>             <td> Reset the I/O statistics
</table>

<a name="misc">
<p>
<table border=0 cellpadding=4 cellspacing=10 width="100%">
<tr><td width="100%" bgcolor="#ccccff"><font face="Arial">
<center><b>Documentation, Testing, Submission, Etc.</b></center>
</font></td></tr>
</table>

As usual: you are expected to include comments in your code; you
should submit a 1-2 page description (in a plain text file
<tt>sm_DOC</tt>) covering your design, key data structures, testing
strategy, and known bugs, and citing any assistance you received; you
will be asked some design questions by email.  Part 3 will be
submitted via the <tt>submit</tt> script in the same way you submitted
Parts 1 and 2.  
As always, please compile using
the <tt>-DPF_STATS</tt> flag, remember to run the "<tt>submit -c</tt>"
script with argument "<tt>3</tt>" (for project part 3), and be sure to
check the file <tt>submit.sm</tt> before issuing the final "<tt>submit
-s 3</tt>" command. Your submission should include executables for
<code>dbcreate</code>, <code>dbdestrory</code>, and
<code>redbase</code>, which should be the result of running
"<code>make</code>" from within your submission directory.

<p>Because the SM component enables you to work with the interactive
RedBase prompt and submit user-level commands, a separate test program
is not required.  However, you will need to create ASCII data files in
order to bulk-load your relations, and you may find it convenient to
use scripts for testing.  There are some data files located in
directory <tt>/usr/class/cs346/redbase/data/</tt> that you may use if
you like, but we strongly encourage you to create your own additional
data files.  We also have provided an initial test shell script
<tt>sm_test</tt> and one example test <tt>sm_test.1</tt>, obtained
when you run <tt>setup</tt>.  The example test loads relations from
the data files mentioned above.  As always, you will need to generate
many more tests to thoroughly exercise your code, and we will conduct
tests of our own during the grading process.  You should find that,
beginning with this component, it's much more fun to test your system
since you can now interact with it in user mode.


</BODY>
